Every xdist worker of a backend shard collects the whole shard and, on
a fresh runner, rewrites the asserts of each test file because no
cached pyc exists. A new step before the full-suite test step writes
those pycs once, one process per core, with pytest's own rewrite
functions, under the cache root the rootdir conftest sets. It parses
and compiles source and imports no test module, conftest or kiro_crew,
unlike the serial --collect-only prewarm that measured slower.
The step is best effort: it exits 0 whatever happens and pytest
rewrites any file it did not cache. It does not choose what a shard
collects. This is an experiment with a ship gate, measured on the
pull request's own full-suite run (#39368).
Problem / Motivation
Goal: A full backend CI run spends less time collecting tests before the first test starts.
Every xdist worker on a backend shard collects the whole shard and repeats pytest's assertion rewrite for every test file, because CI starts with an empty bytecode cache. A Linux worker collects for a median 252 s and a Windows worker for 108 s (9 green full-suite runs on main, 2026-11-18). #39341 measured a serial
pytest --collect-onlyprewarm and dropped it: it cost 104-115 s and saved 42-75 s, because it imports every test module in one process.Why it matters
Collection is paid by every worker of all 16 shards on every full run, before any test runs. It is CPU-bound, so it does not shrink when workers are added.
Not a goal
actions/cache) or committing them.--collect-onlyprewarm measured in ci: give Backend Tests (Windows) the reduced and leaf scopes Linux uses #39341.What changed (motivation → approach → change)
Workers repeat the same rewrite → do it once, in parallel, without importing anything → a new step before
Run testsinbackend-testandbackend-test-windows.This is an experiment with a ship gate. From #39368: "Ship only if (prewarm step time + new collection time) is lower than the old collection time on the median Linux shard by at least 30 s." The PR stays a draft until its own full-suite run is measured. Placeholder: the per-shard table (prewarm step time and
collection_end.elapsed, 8 Linux + 8 Windows shards, against the median of the last 3 green full-suite runs on main) will be added as a PR comment after CI. If the gate fails, this PR is closed with the numbers.scripts/ci_rewrite_prewarm.py(new). Takes--file-shards,--file-shardand--file-shard-pins, as the test step does. Splits the files over onespawnprocess per core, largest first.steps.scope.outputs.reduced == 'false'), with the test step's shell, so the same user and home. TheRun testssteps are not edited.continue-on-errorand a 5-minute timeout. A file it did not cache is rewritten by pytest.testpaths, matchingpython_files, owner decided byscripts.ci_file_shards.file_shardwith the same pin file. Plus everyconftest.pyunder the rootdir andtestpathspython_files(aiohttp/test_utils.py),pytest_plugins,register_assert_rewrite. Finding them needs an import. pytest rewrites them as before (70 files on shard 1)_rewrite_test,_write_pyc,PYC_TAIL(<name>.cpython-312-pytest-9.0.3.pyc),get_cache_dirpycache_prefix()mirrors_redirect_bytecode_cacheintest/conftest_floor_configure.py:$XDG_CACHE_HOMEor<account home>/.cache, thenkirocrew/pycache. Each process setssys.pycache_prefixto it, soget_cache_dirbuilds the path, drive letter handling included. A test pins the two functions equalos.getcwd(), as pytest uses, since the cache directory is derived from the absolute source pathLocal evidence only (dev host, 32 cores, loaded; shard 1 of 8, 992 files, 33,590 tests;
pytest --collect-only -q -n0 --no-cov, fresh cache root per arm):With 32 processes the prewarm took 2.0 s. In a traced run after a prewarm (
--debug), pytest reported "found cached rewritten pyc" for 985 modules and "rewriting" for 70, none of them prewarmed files. A 2-worker xdist collection after a prewarm left all 992 prewarmed pycs untouched and added the same 70, so workers read the cache too. These numbers are one process without coverage on a developer host. They do not predict the CI result.Backwards compatibility
Compatible: one added CI step that only writes bytecode under the runner's cache directory; no product code moves.
Removes nothing: the test steps, their scopes and the files each shard collects are unchanged.
Tests
test/test_ci_rewrite_prewarm.py(new, 11 tests):--debug: the rewriter traces "found cached rewritten pyc" for every test file and "rewriting" for none, and every pyc, conftest ones included, is byte- and mtime-identical afterwards.sys.modulesholds nokiro_crew,conftestor test module, and the markers do appear when pytest runs.file_shardand a pin.python_files,testpathsand pytest's defaultnorecursedirs._redirect_bytecode_cachesets.Run tests, full suite only, same shell, same shard flags and pin file,continue-on-error, noactions/cachestep, and no other job runs it.Existing pins re-run green:
test/test_ci_file_shards.py,test/test_ci_pytest_progress.py,test/test_ci_windows_canary.py.Manual verification
Ran the script on this checkout's shard 1 with a fresh cache root, then collected the shard (table above). Not run on Windows locally: the Windows path shape and
spawnare exercised by this PR's Windows shards, which run the new test file and the new step.Real-machine verification
N/A -- CI workflow only; measured on CI runners.
Related Issues
Closes #39368
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commitsContribution License Agreement
Contributed under the project's Apache-2.0 license.
AI-assisted, opened by Claude Code